Competency-based classification of COMLEX-USA cognitive examination test items.

CONTEXT The Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) currently assesses osteopathic medical knowledge via a series of 3 progressive cognitive examinations and 1 clinical skills assessment. In 2009, the National Board of Osteopathic Medical Examiners created the Fundamental Osteopathic Medical Competencies (FOMC) document to outline the essential competencies required for the practice of osteopathic medicine. OBJECTIVES To measure the distribution and extent to which cognitive examination items of the current series of COMLEX-USA assess knowledge of each of the medical competencies included in the FOMC document. METHODS Eight graduate medical education panelists with expertise in competency-based assessment reviewed 1046 multiple-choice examination items extracted from the 3 COMLEX-USA cognitive examinations (Level 1, Level 2-Cognitive Evaluation, and Level 3) used during the 2008-2009 testing cycle. The 8 panelists individually judged each item to classify it as 1 of the 6 fundamental osteopathic medical competencies described in the FOMC document. RESULTS Panelists made 8368 judgments. The majority of the sample examination items were classified as either patient care (3343 [40%]) or medical knowledge (4236 [51%]). Panelists also reported these 2 competencies as being the easiest to define, teach, and assess. The frequency of medical knowledge examination items decreased throughout the COMLEX-USA series (69%, 43%, 40%); conversely, items classified as interpersonal and communication skills, systems-based practice, practice-based learning and improvement, and professionalism increased throughout the 3-examination series. CONCLUSION Results indicate that knowledge of each of the 6 competencies is being assessed to some extent with the current COMLEX-USA format. These findings provide direction for the enhancement of existing examinations and development of new assessment tools.